Skip to content

wd_rtsp.sh: read process, socket and CPU state from /proc - #489

Closed
grzesiczek1 wants to merge 1 commit into
alienatedsec:masterfrom
grzesiczek1:fix/wd-rtsp-proc
Closed

grzesiczek1 wants to merge 1 commit into
alienatedsec:masterfrom
grzesiczek1:fix/wd-rtsp-proc

Conversation

@grzesiczek1

Copy link
Copy Markdown
Contributor

Follow-up to point 1 of #485; the same kind of change was merged for wd.sh in
roleoroleo/yi-hack-Allwinner-v2#1200. Every 10 s wd_rtsp.sh runs ps three times, top twice
and netstat once, each through grep and awk pipelines, plus get_config. In a test
fixture that is 34–35 new processes per healthy loop, on a single-core camera.

This PR reads the same state with shell builtins:

  • Processes: one pass over /proc/*/comm per loop finds rRTSPServer and counts
    h264grabber, rmm, cloud and mqttv4. The h264grabber count keeps the meaning it
    has since wd_rtsp.sh: restart grabbers when an expected one is missing #483.
  • Sockets: established connections on the RTSP port come from /proc/net/tcp and
    /proc/net/tcp6, the
    condition wd_rtsp.sh: make the locked rRTSPServer check work #482 checks with netstat.
  • Locked rRTSPServer: the check compares utime + stime from /proc/PID/stat over a
    sample time, instead of reading 0.0 in top's %CPU column. top -n 1 takes that
    column from the same counters over 0.1 s. The first sample takes 1 s; the nine that
    confirm a lock take 0.3 s each (1 s if sleep takes no fractions). As before, this runs
    only while a client is connected.
  • DISABLE_CLOUD is still read from system.conf in every loop, with read instead
    of grep | cut | awk.

The actions, their order and the log messages are unchanged. Three details differ:

  1. The process names are compared exactly. ps | grep rmm also matched any other command
    line containing rmm, which would hide a missing rmm.
  2. After check_grabber restarts the grabbers and rRTSPServer, check_rtsp waits for the
    next loop, so it does not judge the PID it saw before the restart.
  3. While a possible lock is being counted (COUNTER > 0) the loop has no sleep. If the
    clients disconnect at that point, the counter is never reset and the loop spins until a
    client connects again. The loop now pauses 1 s whenever it took no CPU sample.

Tests

Both versions ran under QEMU with the BusyBox 1.36.1 of a camera running 0.4.1, in their
own PID and network namespace. The daemons are stand-in processes with the right names and
real local TCP sockets. The recovery commands log the action and then really do it on the
stand-ins, so a recovery brings the fixture back to a healthy state. reboot only logs.

31 checks pass:

  • nothing but the start message when everything runs, with and without clients, and with
    cloud missing while DISABLE_CLOUD=no;
  • the same log lines and kills in the same order as master, each recovery done exactly
    once, for: rRTSPServer missing, one of three h264grabber missing, rmm missing,
    cloud missing with DISABLE_CLOUD=yes, mqttv4 missing, a locked rRTSPServer with
    clients, and RTSP=no.
master this PR
processes started per healthy loop 35.3 0
the same with two clients connected 33.7 0
loops in the 10 s after the clients of a suspected server disconnect 25 10
restart of a locked rRTSPServer (ten counts) 3.8 s 4.2 s
false "possible locked process" lines in 30 s for a server that uses the CPU in bursts 6 0

Master takes ten 0.1-second samples, so it also suspects a server that was merely idle
for a moment. Here the first sample is one second long, and a restart needs ten samples
in a row without a single CPU tick (3.7 s of sampling). Both restart times come from a fast host; on the camera
the pipelines of master take longer.

On a Yi Outdoor (3.0.0.0A, 0.4.1) a copy of this script with the actions replaced by log
lines ran for 24 hours next to the real watchdog while an NVR recorded over RTSP/TCP: about
7,800 loops, no intended actions (no restart, no "possible locked"), and 692 s of CPU time
for the script itself, 0.8 % of the single core.

🤖 Generated with Claude Code

Every 10 s the watchdog ran ps, top and netstat through grep and awk
pipelines, about 35 processes per loop on a single-core camera.

The checks now use shell builtins only:

- one pass over /proc/*/comm finds rRTSPServer and counts h264grabber,
  rmm, cloud and mqttv4;
- established connections on the RTSP port come from /proc/net/tcp and
  /proc/net/tcp6;
- a locked rRTSPServer is detected from utime + stime in /proc/PID/stat
  instead of "0.0" in top's %CPU column (which top takes from the same
  counters over 0.1 s). The first sample takes 1 s, the nine that confirm
  a lock 0.3 s each;
- DISABLE_CLOUD is read from system.conf with read.

The actions, their order and the log messages are unchanged. Three
details differ:

- process names are compared exactly; ps | grep rmm also matched any
  other command line containing "rmm", which could hide a missing rmm;
- after check_grabber restarts the grabbers and rRTSPServer, check_rtsp
  is skipped until the next loop, so it does not judge the old PID;
- while a possible lock is being counted the loop had no sleep at all. It
  now pauses 1 s when no CPU sample was taken in that loop.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
alienatedsec added a commit that referenced this pull request Nov 16, 2026
wd_rtsp.sh: read process, socket and CPU state from /proc

Conflicts resolved: kept our MQTT=no guard in check_mqttv4(), which now
uses the new /proc-based N_MQTTV4 counter instead of a ps|grep pipeline.
@alienatedsec

Copy link
Copy Markdown
Owner

Hi @grzesiczek1 I already merged your pull requests to release-0.4.2 branch as there were some conflicts with my recent changes. Thank you for the contribution. Once tested, it will be merged back to master and the new version will be released.

alienatedsec pushed a commit that referenced this pull request Nov 17, 2026
The MQTT check added with #489 calls get_config in every loop, which
runs a subshell plus grep, cut and awk: 4 processes every 10 seconds,
also when MQTT is off. Use the read_conf helper from #489 instead, as
check_cloud does for DISABLE_CLOUD. The setting is still read in every
loop, and mqttv4 is only checked when MQTT=yes.

The value is now quoted. Before, with no MQTT line in system.conf (or an
empty value) the unquoted empty expansion made the test fail with
"unknown operand", the guard did not return, and mqttv4 was restarted
anyway.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants